Pre-operative planning of bone graft to be harvested from donor site

ABSTRACT

Techniques are described for bone graft selection in orthopedic surgery. Processing circuitry may determine a bone graft template for a bone graft to be connected to a first anatomical object, determine information indicative of placement of the bone graft template within a representation of a second anatomical object based on image data for one or more images of anatomical objects, and output the information indicative of the placement of the bone graft template within the representation of the second anatomical object.

This patent application claims the benefit of U.S. Provisional PatentApplication 62/884,079, filed Aug. 7, 2019, the entire content of whichis incorporated by reference.

BACKGROUND

Surgical repair procedures involve repair and/or replacement of adamaged or diseased anatomical object. One example of surgical repairprocedures is bone grafting where a surgeon harvests bone, referred toas a bone graft, from a donor site for repairing damaged or diseasedbone or for conditioning the damaged or diseased bone for implantation.The process of removing of the bone graft can negatively impact thedonor site, such as causing bone fractures, cosmetic deformities, orinjuring surrounding tissue, especially where the bone graft is largerthan an amount of bone the donor site can donate without negative impactto the donor site. Also, after bone graft removal, the donor site maynot have enough remaining bone to ensure proper fixation of an implantin the case an implant is needed at the donor site after bone graftharvesting procedure.

SUMMARY

This disclosure describes example techniques for providing preoperativeplanning for determining characteristics of a donor site for donating abone graft. A computing device may determine a bone graft template ofthe bone graft (e.g., based on user input or determined based on imageprocessing). A display device may display the bone graft template on arepresentation of the donor site, and a surgeon may determine whetherthe donor site includes sufficient bone to donate for the bone graft. Insome examples, a computing device may be configured to compare the bonegraft template to representation of the donor site, and outputinformation indicative of whether the donor site includes sufficientbone to donate for the bone graft.

In this manner, the example techniques provide a technical solution fordetermining whether donor site is a proper donor site prior to a surgeonperforming surgery. For instance, the example techniques provide forpractical applications of preoperative planning utilizing imageprocessing for determining whether to extract a bone graft from a donorsite. In some examples, the disclosure describes example techniques todetermine information indicative of a manner in which the bone graft isto be extracted (e.g., output visual information showing axis of cut,output visual information showing maximum depth of cut, outputinformation indicative of type of tool to use, and the like). With theinformation indicative of the manner in which the bone graft is to beextracted, it may be possible for the computing device or the surgeon todetermine a tools to use for the surgery including patient specifictools that can be viewed using augmented reality or mixed reality, andthe like.

In one example, the disclosure describes a system for bone graftselection in orthopedic surgery, the system comprising a memoryconfigured to store image data for one or more images of anatomicalobjects and processing circuitry. The processing circuitry is configuredto determine a bone graft template for a bone graft to be connected to afirst anatomical object, determine information indicative of placementof the bone graft template within a representation of a secondanatomical object based on the image data, and output the informationindicative of the placement of the bone graft template within therepresentation of the second anatomical object.

In one example, the disclosure describes a method for bone graftselection in orthopedic surgery, the method comprising determining abone graft template for a bone graft to be connected to a firstanatomical object, determining information indicative of placement ofthe bone graft template within a representation of a second anatomicalobject based on image data for one or more images of anatomical objects,and outputting the information indicative of the placement of the bonegraft template within the representation of the second anatomicalobject.

In one example, the disclosure describes a system for bone graftselection in orthopedic surgery, the system comprising means fordetermining a bone graft template for a bone graft to be connected to afirst anatomical object, means for determining information indicative ofplacement of the bone graft template within a representation of a secondanatomical object based on image data for one or more images ofanatomical objects, and means for outputting the information indicativeof the placement of the bone graft template within the representation ofthe second anatomical object.

In one example, the disclosure describes a computer-readable storagemedium storing instructions that when executed cause one or moreprocessors to determine a bone graft template for a bone graft to beconnected to a first anatomical object, determine information indicativeof placement of the bone graft template within a representation of asecond anatomical object based on image data for one or more images ofanatomical objects, and output the information indicative of theplacement of the bone graft template within the representation of thesecond anatomical object.

The details of various examples of the disclosure are set forth in theaccompanying drawings and the description below. Various features,objects, and advantages will be apparent from the description, drawings,and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an orthopedic surgical system according toan example of this disclosure.

FIG. 2 is a block diagram of an orthopedic surgical system that includesa mixed reality (MR) system, according to an example of this disclosure.

FIG. 3 is a block diagram illustrating an example of computing systemconfigured to perform one or more examples described in this disclosure.

FIGS. 4 and 5 are conceptual diagrams illustrating examples of boneextraction for bone grafting.

FIG. 6 is a conceptual diagram illustrating an example of the extractedbone grafted to glenoid bone.

FIGS. 7A and 7B are conceptual diagrams illustrating examples of a bonegraft template within a representation of a donor site.

FIG. 8 is a flowchart illustrating example methods of operations inaccordance with one or more example techniques described in thisdisclosure.

DETAILED DESCRIPTION

Orthopedic surgery can involve implanting one or more prosthetic devicesto repair or replace a patient's damaged or diseased joint. Orthopedicsurgery also includes grafting bone from a donor site to a target bone.The target bone may be a damaged or diseased bone or may be bone that isoffset or augmented using the graft. Autografting refers to bone beingtaken from a donor site within the patient for grafting to the targetbone within the same patient. The example techniques are described withrespect to autografting but are applicable to other types of grafting aswell (e.g., from a donor site not within the patient so calledallograft, or from a synthetic source of material so called syntheticbone graft substitute). In some examples, the bone grafting may be usedto augment or otherwise prepare the damaged or diseased bone forreceiving the one or more prosthetic devices. In some examples, the bonegraft is attached to a prosthetic device, and then the combined assembly(e.g., prosthetic device plus bone graft) are attached to the targetbone.

Virtual surgical planning tools may use image data of the diseased ordamaged joint to generate an accurate three-dimensional bone model thatcan be viewed and manipulated preoperatively by the surgeon. These toolscan enhance surgical outcomes by allowing the surgeon to simulate thesurgery, select or design an implant that more closely matches thecontours of the patient's actual bone, and select or design surgicalinstruments and guide tools that are adapted specifically for repairingthe bone of a particular patient.

Use of these planning tools typically results in generation of apreoperative surgical plan, complete with an implant and surgicalinstruments that are selected or manufactured for the individualpatient. Oftentimes, once in the actual operating environment, thesurgeon may desire to verify the preoperative surgical planintraoperatively relative to the patient's actual bone. Thisverification may result in a determination that an adjustment to thepreoperative surgical plan is needed, such as a different implant, adifferent positioning or orientation of the implant, and/or a differentsurgical guide for carrying out the surgical plan. In addition, asurgeon may want to view details of the preoperative surgical planrelative to the patient's real bone during the actual procedure in orderto more efficiently and accurately position and orient the implantcomponents. For example, the surgeon may want to obtain intra-operativevisualization that provides guidance for positioning and orientation ofimplant components, guidance for preparation of bone or tissue toreceive the implant components, guidance for reviewing the details of aprocedure or procedural step, and/or guidance for selection of tools orimplants and tracking of surgical procedure workflow.

As described above, for bone grafting, the surgeon extracts a bone graftfrom a donor site for grafting to the target bone. The surgeon maydesire to determine characteristics of the donor site to determinewhether the donor site is adequate for providing the bone graft. Forexample, the surgeon may desire to determine whether there is sufficientbone at the donor site, and as another example, whether the bone at thedonor site is sufficiently dense. There may potentially be some negativeimpacts if there is insufficient bone at the donor site and/or if thebone at donor site is not sufficiently dense. For instance, there ispotentially an increase in the probability of fracture at the donorsite, there may be cosmetic deformities, and there may be damage tosurrounding tissue at the donor site because the bone at the donor siteis insufficient to properly support the tissue.

In accordance with the example techniques described in this disclosure,a computing device may generate information indicative of whether adonor site is suitable for being a donor for a bone graft. The surgeonmay utilize the generated information to determine preoperativelywhether to extract bone from the donor site. The surgeon may alsoutilize the generated information intraoperatively to confirm that thedonor site is suitable, and as described in more detail, possiblyutilize the generated information intraoperatively for surgicalguidance.

For example, processing circuitry (e.g., processing circuitry of one ormore computing devices) may be configured to determine a bone grafttemplate for a bone graft to be connected to a first anatomical object(e.g., target bone). Bone graft template refers to a representation ofsize and shape of the bone graft that is to be extracted. The bone grafttemplate may be a graphical representation that can be displayed. Thebone graft template may be represented by shape equations that define aparticular size and shape, or points within particular size and shapehaving assigned coordinates, as a few examples.

There may be various ways in which the processing circuitry determinesthe bone graft template. As one example, the processing circuitry mayoutput for display image data showing the target bone, and theprocessing circuitry may receive input from the surgeon for what thebone graft template should look like. The processing circuitry maydetermine the bone graft template based on the received input. Asanother example, the processing circuitry may be configured to generatea pre-morbid construction of the target bone. The processing circuitrymay determine a difference between the pre-morbid construction and theactual target bone to determine the bone graft template.

The processing circuitry may determine information indicative ofplacement of the bone graft template within a representation of a secondanatomical object (e.g., donor site). For example, a memory may storeimage data for one or more images of anatomical objects. As one example,memory may store image data of computerized tomography (CT) scans of thepatient. Part of the image data includes a representation of the donorsite (e.g., images of the donor site). The processing circuitry maydetermine information indicative of how the bone graft template fitswithin the representation of the donor site based on the image data.

As one example, the surgeon may move (e.g., drag and place with astylus) a displayed representation of the bone graft template within therepresentation of the donor site. In response to the movement of thebone graft template, the processing circuitry may determine informationneeded to move the representation of the bone graft template (e.g.,information such as coordinates of where the bone graft template is tobe displayed). The processing circuitry may then output informationindicative of the placement of the bone graft template within therepresentation of the donor site (e.g., output graphical informationused to render the bone graft template with the representation of thedonor site).

As another example, the processing circuitry may be configured toutilize the points or shape equations of the bone graft template and thepoints in the representation of the donor site to determine how to placethe bone graft template within the representation of the donor site. Theprocessing circuitry may utilize certain criteria in determininginformation indicative of the placement such as information that definesboundaries within the donor site from where a bone graft can beextracted. For instance, the boundary may define certain configurationsfrom which a bone graft can be extracted with comparative ease, wherefor other configurations of the bone graft template within the donorsite, there may be complications in removing the bone graft (e.g., hardto access the bone graft). As another example, the boundary may defineportions of the donor site where if the bone graft is removed from otherportions there may be a possibility of cosmetic defect, injury, orsusceptibility to injury. The processing circuitry may output thedetermined information (e.g., graphical information used to render thebone graft template within the representation of the donor site).

In some examples, in addition to generating information that can be usedto determine whether a donor site is suitable for extracting a bonegraft, the processing circuitry may be configured to generateinformation indicating whether the donor site is potentially suitable asa donor site for the bone graft (e.g., whether there is sufficientclearance, whether the bone graft extraction process can be performedwith relative surgical ease, and the like). Also, in some examples, theprocessing circuitry may generate pre-operative planning information.

As one example, the processing circuitry may generate informationindicative of a location from which the bone graft is to be extractedwithin the donor site (e.g., location that is to be cut for extractingthe bone graft). In some examples, processing circuitry is configured togenerate information indicative of an axis along which to the secondanatomical object for extracting the bone graft. The processingcircuitry may be configured to generate information indicative of amaximum depth at which to cut the second anatomical object forextracting the bone graft. As one example, the processing circuitry isconfigured to generate information indicative of a types of tool toutilize to cut the second anatomical object for extracting the bonegraft.

In the techniques described in this disclosure, there may be variousways in which the surgeon may preoperatively view image content such asbone graft template, placement of bone graft template in donor site, andsurgical guidance information. Also, in some examples, the surgeon maybe able to view the bone graft template, placement of bone grafttemplate in donor site, and surgical guidance information during theoperation.

For example, the surgeon may use a mixed reality (MR) visualizationsystem to assist with creation, implementation, verification, and/ormodification of a surgical plan before and during a surgical procedure.Because MR, or in some instances virtual reality (VR), may be used tointeract with the surgical plan, this disclosure may also refer to thesurgical plan as a “virtual” surgical plan. Visualization tools otherthan or in addition to mixed reality visualization systems may be usedin accordance with techniques of this disclosure.

A surgical plan, e.g., as generated by the BLUEPRINT™ system or anothersurgical planning platform, may include information defining a varietyof features of a surgical procedure, such as features of particularsurgical procedure steps to be performed on a patient by a surgeonaccording to the surgical plan including, for example, bone or tissuepreparation steps and/or steps for selection, modification and/orplacement of implant components. Such information may include, invarious examples, dimensions, shapes, angles, surface contours, and/ororientations of implant components to be selected or modified bysurgeons, dimensions, shapes, angles, surface contours and/ororientations to be defined in bone or tissue by the surgeon in bone ortissue preparation steps, and/or positions, axes, planes, angle and/orentry points defining placement of implant components by the surgeonrelative to patient bone or tissue. Information such as dimensions,shapes, angles, surface contours, and/or orientations of anatomicalfeatures of the patient may be derived from imaging (e.g., x-ray, CT,MM, ultrasound or other images), direct observation, or othertechniques.

In this disclosure, the term “mixed reality” (MR) refers to thepresentation of virtual objects such that a user sees images thatinclude both real, physical objects and virtual objects. Virtual objectsmay include text, 2-dimensional surfaces, 3-dimensional models, or otheruser-perceptible elements that are not actually present in the physical,real-world environment in which they are presented as coexisting. Inaddition, virtual objects described in various examples of thisdisclosure may include graphics, images, animations or videos, e.g.,presented as 3D virtual objects or 2D virtual objects. Virtual objectsmay also be referred to as virtual elements. Such elements may or maynot be analogs of real-world objects. In some examples, in mixedreality, a camera may capture images of the real world and modify theimages to present virtual objects in the context of the real world. Insuch examples, the modified images may be displayed on a screen, whichmay be head-mounted, handheld, or otherwise viewable by a user.

This type of mixed reality is increasingly common on smartphones, suchas where a user can point a smartphone's camera at a sign written in aforeign language and see in the smartphone's screen a translation in theuser's own language of the sign superimposed on the sign along with therest of the scene captured by the camera. In some examples, in mixedreality, see-through (e.g., transparent) holographic lenses, which maybe referred to as waveguides, may permit the user to view real-worldobjects, i.e., actual objects in a real-world environment, such as realanatomy, through the holographic lenses and also concurrently viewvirtual objects.

The Microsoft HOLOLENS™ headset, available from Microsoft Corporation ofRedmond, Wash., is an example of a MR device that includes see-throughholographic lenses, sometimes referred to as waveguides, that permit auser to view real-world objects through the lens and concurrently viewprojected 3D holographic objects. The Microsoft HOLOLENS™ headset, orsimilar waveguide-based visualization devices, are examples of an MRvisualization device that may be used in accordance with some examplesof this disclosure. Some holographic lenses may present holographicobjects with some degree of transparency through see-through holographiclenses so that the user views real-world objects and virtual,holographic objects. In some examples, some holographic lenses may, attimes, completely prevent the user from viewing real-world objects andinstead may allow the user to view entirely virtual environments. Theterm mixed reality may also encompass scenarios where one or more usersare able to perceive one or more virtual objects generated byholographic projection. In other words, “mixed reality” may encompassthe case where a holographic projector generates holograms of elementsthat appear to a user to be present in the user's actual physicalenvironment.

In some examples, in mixed reality, the positions of some or allpresented virtual objects are related to positions of physical objectsin the real world. For example, a virtual object may be tethered to atable in the real world, such that the user can see the virtual objectwhen the user looks in the direction of the table but does not see thevirtual object when the table is not in the user's field of view. Insome examples, in mixed reality, the positions of some or all presentedvirtual objects are unrelated to positions of physical objects in thereal world. For instance, a virtual item may always appear in the topright of the user's field of vision, regardless of where the user islooking.

Augmented reality (AR) is similar to MR in the presentation of bothreal-world and virtual elements, but AR generally refers topresentations that are mostly real, with a few virtual additions to“augment” the real-world presentation. For purposes of this disclosure,MR is considered to include AR. For example, in AR, parts of the user'sphysical environment that are in shadow can be selectively brightenedwithout brightening other areas of the user's physical environment. Thisexample is also an instance of MR in that the selectively-brightenedareas may be considered virtual objects superimposed on the parts of theuser's physical environment that are in shadow.

Furthermore, in this disclosure, the term “virtual reality” (VR) refersto an immersive artificial environment that a user experiences throughsensory stimuli (such as sights and sounds) provided by a computer.Thus, in virtual reality, the user may not see any physical objects asthey exist in the real world. Video games set in imaginary worlds are acommon example of VR. The term “VR” also encompasses scenarios where theuser is presented with a fully artificial environment in which somevirtual object's locations are based on the locations of correspondingphysical objects as they relate to the user. Walk-through VR attractionsare examples of this type of VR.

The term “extended reality” (XR) is a term that encompasses a spectrumof user experiences that includes virtual reality, mixed reality,augmented reality, and other user experiences that involve thepresentation of at least some perceptible elements as existing in theuser's environment that are not present in the user's real-worldenvironment. Thus, the term “extended reality” may be considered a genusfor MR and VR. XR visualizations may be presented in any of thetechniques for presenting mixed reality discussed elsewhere in thisdisclosure or presented using techniques for presenting VR, such as VRgoggles.

FIG. 1 is a block diagram of an orthopedic surgical system 100 accordingto an example of this disclosure. Orthopedic surgical system 100includes a set of subsystems. In the example of FIG. 1, the subsystemsinclude a virtual planning system 102, a planning support system 104, amanufacturing and delivery system 106, an intraoperative guidance system108, a medical education system 110, a monitoring system 112, apredictive analytics system 114, and a communications network 116. Inother examples, orthopedic surgical system 100 may include more, fewer,or different subsystems. For example, orthopedic surgical system 100 mayomit medical education system 110, monitor system 112, predictiveanalytics system 114, and/or other subsystems. In some examples,orthopedic surgical system 100 may be used for surgical tracking, inwhich case orthopedic surgical system 100 may be referred to as asurgical tracking system. In other cases, orthopedic surgical system 100may be generally referred to as a medical device system.

Users of orthopedic surgical system 100 may use virtual planning system102 to plan orthopedic surgeries. Users of orthopedic surgical system100 may use planning support system 104 to review surgical plansgenerated using orthopedic surgical system 100. Manufacturing anddelivery system 106 may assist with the manufacture and delivery ofitems needed to perform orthopedic surgeries. Intraoperative guidancesystem 108 provides guidance to assist users of orthopedic surgicalsystem 100 in performing orthopedic surgeries. Medical education system110 may assist with the education of users, such as healthcareprofessionals, patients, and other types of individuals. Pre- andpostoperative monitoring system 112 may assist with monitoring patientsbefore and after the patients undergo surgery. Predictive analyticssystem 114 may assist healthcare professionals with various types ofpredictions. For example, predictive analytics system 114 may applyartificial intelligence techniques to determine a classification of acondition of an orthopedic joint, e.g., a diagnosis, determine whichtype of surgery to perform on a patient and/or which type of implant tobe used in the procedure, determine types of items that may be neededduring the surgery, and so on.

The subsystems of orthopedic surgical system 100 (i.e., virtual planningsystem 102, planning support system 104, manufacturing and deliverysystem 106, intraoperative guidance system 108, medical education system110, pre- and postoperative monitoring system 112, and predictiveanalytics system 114) may include various systems. The systems in thesubsystems of orthopedic surgical system 100 may include various typesof computing systems, computing devices, including server computers,personal computers, tablet computers, smartphones, display devices,Internet of Things (IoT) devices, visualization devices (e.g., mixedreality (MR) visualization devices, virtual reality (VR) visualizationdevices, holographic projectors, or other devices for presentingextended reality (XR) visualizations), surgical tools, and so on. Aholographic projector, in some examples, may project a hologram forgeneral viewing by multiple users or a single user without a headset,rather than viewing only by a user wearing a headset. For example,virtual planning system 102 may include a MR visualization device andone or more server devices, planning support system 104 may include oneor more personal computers and one or more server devices, and so on. Acomputing system is a set of one or more computing systems configured tooperate as a system. In some examples, one or more devices may be sharedbetween the two or more of the subsystems of orthopedic surgical system100. For instance, in the previous examples, virtual planning system 102and planning support system 104 may include the same server devices.

In the example of FIG. 1, the devices included in the subsystems oforthopedic surgical system 100 may communicate using communicationnetwork 116. Communication network 116 may include various types ofcommunication networks including one or more wide-area networks, such asthe Internet, local area networks, and so on. In some examples,communication network 116 may include wired and/or wirelesscommunication links.

Many variations of orthopedic surgical system 100 are possible inaccordance with techniques of this disclosure. Such variations mayinclude more or fewer subsystems than the version of orthopedic surgicalsystem 100 shown in FIG. 1. For example, FIG. 2 is a block diagram of anorthopedic surgical system 200 that includes one or more mixed reality(MR) systems, according to an example of this disclosure. Orthopedicsurgical system 200 may be used for creating, verifying, updating,modifying and/or implementing a surgical plan. In some examples, thesurgical plan can be created preoperatively, such as by using a virtualsurgical planning system (e.g., the BLUEPRINT™ system), and thenverified, modified, updated, and viewed intraoperatively, e.g., using MRvisualization of the surgical plan. In other examples, orthopedicsurgical system 200 can be used to create the surgical plan immediatelyprior to surgery or intraoperatively, as needed. In some examples,orthopedic surgical system 200 may be used for surgical tracking, inwhich case orthopedic surgical system 200 may be referred to as asurgical tracking system. In other cases, orthopedic surgical system 200may be generally referred to as a medical device system.

In the example of FIG. 2, orthopedic surgical system 200 includes apreoperative surgical planning system 202, a healthcare facility 204(e.g., a surgical center or hospital), a storage system 206 and anetwork 208 that allows a user at healthcare facility 204 to accessstored patient information, such as medical history, image datacorresponding to the damaged joint or bone and various parameterscorresponding to a surgical plan that has been created preoperatively(as examples). Preoperative surgical planning system 202 may beequivalent to virtual planning system 102 of FIG. 1 and, in someexamples, may generally correspond to a virtual planning system similaror identical to the BLUEPRINT™ system.

In the example of FIG. 2, healthcare facility 204 includes a mixedreality (MR) system 212. In some examples of this disclosure, MR system212 includes one or more processing device(s) (P) 210 to providefunctionalities that will be described in further detail below.Processing device(s) 210 may also be referred to as processor(s) orprocessing circuitry. In addition, one or more users of MR system 212(e.g., a surgeon, nurse, or other care provider) can use processingdevice(s) (P) 210 to generate a request for a particular surgical planor other patient information that is transmitted to storage system 206via network 208. In response, storage system 206 returns the requestedpatient information to MR system 212. In some examples, the users canuse other processing device(s) to request and receive information, suchas one or more processing devices that are part of MR system 212, butnot part of any visualization device, or one or more processing devicesthat are part of a visualization device (e.g., visualization device 213)of MR system 212, or a combination of one or more processing devicesthat are part of MR system 212, but not part of any visualizationdevice, and one or more processing devices that are part of avisualization device (e.g., visualization device 213) that is part of MRsystem 212.

In some examples, multiple users can simultaneously use MR system 212.For example, MR system 212 can be used in a spectator mode in whichmultiple users each use their own visualization devices so that theusers can view the same information at the same time and from the samepoint of view. In some examples, MR system 212 may be used in a mode inwhich multiple users each use their own visualization devices so thatthe users can view the same information from different points of view.

In some examples, processing device(s) 210 can provide a user interfaceto display data and receive input from users at healthcare facility 204.Processing device(s) 210 may be configured to control visualizationdevice 213 to present a user interface. Furthermore, processingdevice(s) 210 may be configured to control visualization device 213 topresent virtual images, such as 3D virtual models, 2D images, and so on.Processing device(s) 210 can include a variety of different processingor computing devices, such as servers, desktop computers, laptopcomputers, tablets, mobile phones and other electronic computingdevices, or processors within such devices. In some examples, one ormore of processing device(s) 210 can be located remote from healthcarefacility 204. In some examples, processing device(s) 210 reside withinvisualization device 213. In some examples, at least one of processingdevice(s) 210 is external to visualization device 213. In some examples,one or more processing device(s) 210 reside within visualization device213 and one or more of processing device(s) 210 are external tovisualization device 213.

In the example of FIG. 2, MR system 212 also includes one or more memoryor storage device(s) (M) 215 for storing data and instructions ofsoftware that can be executed by processing device(s) 210. Theinstructions of software can correspond to the functionality of MRsystem 212 described herein. In some examples, the functionalities of avirtual surgical planning application, such as the BLUEPRINT™ system,can also be stored and executed by processing device(s) 210 inconjunction with memory storage device(s) (M) 215. For instance, memoryor storage system 215 may be configured to store data corresponding toat least a portion of a virtual surgical plan. In some examples, storagesystem 206 may be configured to store data corresponding to at least aportion of a virtual surgical plan. In some examples, memory or storagedevice(s) (M) 215 reside within visualization device 213. In someexamples, memory or storage device(s) (M) 215 are external tovisualization device 213. In some examples, memory or storage device(s)(M) 215 include a combination of one or more memory or storage deviceswithin visualization device 213 and one or more memory or storagedevices external to the visualization device.

Network 208 may be equivalent to network 116. Network 208 can includeone or more wide area networks, local area networks, and/or globalnetworks (e.g., the Internet) that connect preoperative surgicalplanning system 202 and MR system 212 to storage system 206. Storagesystem 206 can include one or more databases that can contain patientinformation, medical information, patient image data, and parametersthat define the surgical plans.

For example, medical images of the patient's target bone typically aregenerated preoperatively in preparation for an orthopedic surgicalprocedure. The medical images can include images of the relevant bone(s)taken along the sagittal plane and the coronal plane of the patient'sbody. The medical images can include X-ray images, magnetic resonanceimaging (MM) images, computerized tomography (CT) images, ultrasoundimages, and/or any other type of 2D or 3D image that providesinformation about the relevant surgical area. Storage system 206 alsocan include data identifying the implant components selected for aparticular patient (e.g., type, size, etc.), surgical guides selectedfor a particular patient, and details of the surgical procedure, such asentry points, cutting planes, drilling axes, reaming depths, etc.Storage system 206 can be a cloud-based storage system (as shown) or canbe located at healthcare facility 204 or at the location of preoperativesurgical planning system 202 or can be part of MR system 212 orvisualization device (VD) 213, as examples.

MR system 212 can be used by a surgeon before (e.g., preoperatively) orduring the surgical procedure (e.g., intraoperatively) to create,review, verify, update, modify and/or implement a surgical plan. In someexamples, MR system 212 may also be used after the surgical procedure(e.g., postoperatively) to review the results of the surgical procedure,assess whether revisions are required, or perform other postoperativetasks. To that end, MR system 212 may include a visualization device 213that may be worn by the surgeon and (as will be explained in furtherdetail below) is operable to display a variety of types of information,including a 3D virtual image of the patient's diseased, damaged, orpostsurgical joint and details of the surgical plan, such as a 3Dvirtual image of the prosthetic implant components selected for thesurgical plan, 3D virtual images of entry points for positioning theprosthetic components, alignment axes and cutting planes for aligningcutting or reaming tools to shape the bone surfaces, or drilling toolsto define one or more holes in the bone surfaces, in the surgicalprocedure to properly orient and position the prosthetic components,surgical guides and instruments and their placement on the damagedjoint, and any other information that may be useful to the surgeon toimplement the surgical plan. MR system 212 can generate images of thisinformation that are perceptible to the user of the visualization device213 before and/or during the surgical procedure.

In some examples, MR system 212 includes multiple visualization devices(e.g., multiple instances of visualization device 213) so that multipleusers can simultaneously see the same images and share the same 3Dscene. In some such examples, one of the visualization devices can bedesignated as the master device and the other visualization devices canbe designated as observers or spectators. Any observer device can bere-designated as the master device at any time, as may be desired by theusers of MR system 212.

In this way, FIG. 2 illustrates a surgical planning system that includesa preoperative surgical planning system 202 to generate a virtualsurgical plan customized to repair an anatomy of interest of aparticular patient. For example, the virtual surgical plan may include aplan for an orthopedic joint repair surgical procedure, such as one of astandard total shoulder arthroplasty or a reverse shoulder arthroplasty.In this example, details of the virtual surgical plan may includedetails relating to at least one of preparation of glenoid bone orpreparation of humeral bone. In some examples, the orthopedic jointrepair surgical procedure is one of a stemless standard total shoulderarthroplasty, a stemmed standard total shoulder arthroplasty, a stemlessreverse shoulder arthroplasty, a stemmed reverse shoulder arthroplasty,an augmented glenoid standard total shoulder arthroplasty, and anaugmented glenoid reverse shoulder arthroplasty.

The virtual surgical plan may include a 3D virtual model correspondingto the anatomy of interest of the particular patient and a 3D model of aprosthetic component matched to the particular patient to repair theanatomy of interest or selected to repair the anatomy of interest.Furthermore, in the example of FIG. 2, the surgical planning systemincludes a storage system 206 to store data corresponding to the virtualsurgical plan. The surgical planning system of FIG. 2 also includes MRsystem 212, which may comprise visualization device 213. In someexamples, visualization device 213 is wearable by a user. In someexamples, visualization device 213 is held by a user, or rests on asurface in a place accessible to the user. MR system 212 may beconfigured to present a user interface via visualization device 213. Theuser interface is visually perceptible to the user using visualizationdevice 213. For instance, in one example, a screen of visualizationdevice 213 may display real-world images and the user interface on ascreen. In some examples, visualization device 213 may project virtual,holographic images onto see-through holographic lenses and also permit auser to see real-world objects of a real-world environment through thelenses. In other words, visualization device 213 may comprise one ormore see-through holographic lenses and one or more display devices thatpresent imagery to the user via the holographic lenses to present theuser interface to the user.

In some examples, visualization device 213 is configured such that theuser can manipulate the user interface (which is visually perceptible tothe user when the user is wearing or otherwise using visualizationdevice 213) to request and view details of the virtual surgical plan forthe particular patient, including a 3D virtual model of the anatomy ofinterest (e.g., a 3D virtual bone of the anatomy of interest) and a 3Dmodel of the prosthetic component selected to repair an anatomy ofinterest. In some such examples, visualization device 213 is configuredsuch that the user can manipulate the user interface so that the usercan view the virtual surgical plan intraoperatively, including (at leastin some examples) the 3D virtual model of the anatomy of interest (e.g.,a 3D virtual bone of the anatomy of interest). In some examples, MRsystem 212 can be operated in an augmented surgery mode in which theuser can manipulate the user interface intraoperatively so that the usercan visually perceive details of the virtual surgical plan projected ina real environment, e.g., on a real anatomy of interest of theparticular patient. In this disclosure, the terms real and real worldmay be used in a similar manner. For example, MR system 212 may presentone or more virtual objects that provide guidance for preparation of abone surface and placement of a prosthetic implant on the bone surface.Visualization device 213 may present one or more virtual objects in amanner in which the virtual objects appear to be overlaid on an actual,real anatomical object of the patient, within a real-world environment,e.g., by displaying the virtual object(s) with actual, real-worldpatient anatomy viewed by the user through holographic lenses. Forexample, the virtual objects may be 3D virtual objects that appear toreside within the real-world environment with the actual, realanatomical object.

As described above, in some examples, the techniques described in thisdisclosure further provide for ways in which to determine whether bonecan be extracted from a donor site for grafting. For example, inorthopedics, autograft is commonly used for bony reconstruction.Surgeons extract a bone graft from various locations within thepatient's body according to volume and shape of the bone graft neededfor the given bony reconstruction. However, the desired bone graft maybe too large for a donor site and its extraction (e.g., harvesting) maylead to additional complications at the donor site such as bonefractures, cosmetic deformities, injuries to surrounding tissue, and thelike.

As an example, in reverse shoulder arthroplasty, a bone graft isextracted from the humeral head (e.g., the humeral head is the donorsite) and used to augment or reconstruct the glenoid bone (e.g., forpreparation to insert prothesis). This technique is referred to asBIO-RSA (bony increased offset-reverse shoulder arthroplasty). If thebone graft extracted from the humeral head is too large, there may bepossibility that the extraction results in fracture of the tuberosities,rotator cuff injury and/or excessive bone removal that may alter thequality of the fixation of the component for the BIO-RSA (e.g., stem ornucleus inserted into the humerus).

In accordance with one or more examples described in this disclosure,preoperative surgical planning system 202 may be configured to retrieveimage data for one or more images of anatomical objects stored instorage system 206. The images of the anatomical objects includerepresentations (e.g., as image data) of anatomical objects such as afirst anatomical object (e.g., diseased bone) and a second anatomicalobject (e.g., donor site).

Preoperative surgical planning system 202 may determine a bone grafttemplate for a bone graft to be connected to the first anatomicalobject. For example, visualization device 213 may display arepresentation of the first anatomical object. The surgeon, viewing therepresentation of the first anatomical object, may determine the sizeand shape of the bone graft that is to be connected to the firstanatomical object. The surgeon may interact with the displayedrepresentation and draw a bone graft template based on the size andshape of the bone graft. Visualization device 213 may display the bonegraft template with the first anatomical object for the surgeon toconfirm that the bone graft template adequately represents the bonegraft that is to be connected.

Processing device(s) 210 may output information of the bone grafttemplate to preoperative surgical planning system 202. This is oneexample way in which preoperative surgical planning system 202 maydetermine the bone graft template.

In some examples, storage system 206 may store a plurality ofpre-generated bone graft templates of various size and shapes.Visualization device 213 may display the pre-generated bone grafttemplates, and the surgeon may select one of the pre-generated bonegraft templates. Processing device(s) 210 may output information of theselected pre-generated bone graft template to preoperative surgicalplanning system 202. This is another example way in which preoperativesurgical planning system 202 may determine the bone graft template.

In some examples, preoperative surgical planning system 202 may beconfigured to determine the bone graft template for the bone graft, andpossibly with little to no intervention from the surgeon. For example,preoperative surgical planning system 202 may be configured to determinea shape of the first anatomical object. There may be various ways inwhich preoperative surgical planning system 202 may determine the shapeof the first anatomical object, such as by segmenting out the firstanatomical object from the other anatomical objects. Example ways inwhich to segment out the first anatomical object are described in U.S.Provisional Application Ser. Nos. 62/826,119, 62/826,133, 62/826,146,62/826,168, and 62/826,190 all filed on Mar. 29, 2019 and incorporatedby reference in their entirety. There may be other example ways in whichto segment out the first anatomical object, such as in U.S. Pat. No.8,971,606, and incorporated by reference in its entirety.

As one example, for segmenting, preoperative surgical planning system202 may utilize differences in voxel intensities in image data toidentify separation between bony regions and tissue regions to identifythe first anatomical object. As another example, for segmenting,preoperative surgical planning system 202 may utilize closed-surfacefitting (CSF) techniques in which preoperative surgical planning system202 uses a shape model (e.g., predetermined shape like a sphere or ashape based on statistical shape modeling) and expands or constricts theshape model to fit a contour used to identify separation locationsbetween bony regions and tissue or between tissue.

Preoperative surgical planning system 202 may determine a pre-morbidshape of the target bone (e.g., prior to disease or damage in exampleswhere the target bone is for diseased or damaged bone) of the firstanatomical object. Example ways in which to determine the pre-morbidshape of the first anatomical object are described in U.S. ProvisionalApplication Nos. 62/826,172, 62/826,362, and 62/826,410 all filed onMar. 29, 2019, and incorporated by reference in their entirety.

As one example, for determining pre-morbid shape, preoperative surgicalplanning system 202 may align a representation of the first anatomicalobject to coordinates of an SSM of the first anatomical object.Preoperative surgical planning system 202 may deform the SSM todetermine an SSM that registers to the representation of the alignedfirst anatomical object. The version of the SSM that registers to therepresentation of the first anatomical object may be the pre-morbidshape of the target bone.

Preoperative surgical planning system 202 may compare the shape of thefirst anatomical object to the pre-morbid shape of the first anatomicalobject. For example, preoperative surgical planning system 202 maydetermine a difference between the first anatomical object (e.g., howthe first anatomical object appears after disease or damage) and thepre-morbid shape of the first anatomical object (e.g. how the firstanatomical object appeared before disease or damage). Based on thecomparison (e.g., difference), preoperative surgical planning system 202may determine the bone graft template. For instance, preoperativesurgical planning system 202 may determine a bone graft template thathas the approximately the same size and shape as the difference betweenthe shape of the first anatomical object and the pre-morbid shape of thefirst anatomical object.

In one or more examples, preoperative surgical planning system 202 maybe configured to determine information indicative of placement of thebone graft template within a representation of a second anatomicalobject (e.g., donor site) based on the image data. For example, theimage data includes representations of various anatomical objects withinthe patient, such as the humeral head and the humerus, the iliac crest,and the like. Using BLUEPRINT™ or using one or more the segmentationtechniques described in U.S. Provisional Application Ser. Nos.62/826,119, 62/826,133, 62/826,146, 62/826,168, and 62/826,190 all filedon Mar. 29, 2019 or U.S. Pat. No. 8,971,606, visualization device 213may display a representation of the second anatomical object (e.g., a 3Drepresentation of a potential donor site). Although described withrespect to a 3D representation, in some examples, visualization device213 may display 2D scans of the second anatomical object.

Using visualization device 213, the surgeon may “drag and drop” the bonegraft template (e.g., as drawn by the surgeon or as determined bypreoperative surgical planning system 202) include the representation ofthe second anatomical object. In some examples, the surgeon maytranslate or rotate the bone graft template along the x, y, and/or zaxis before or after dragging and dropping the bone graft template inthe representation of the second anatomical object.

In some examples, preoperative surgical planning system 202 may beconfigured to perform the calculations of rotating the bone grafttemplate and calculating the coordinates of the bone graft template foraligning the bone graft template to the coordinate space of therepresentation of the second anatomical object. For example, the bonegraft template and the representation of the second anatomical objectmay be in different coordinate systems, and to move the bone grafttemplate to the representation of the second anatomical object (e.g.,donor site), preoperative surgical planning system 202 may determine atransformation matrix that provides for rotation, translation, scaling,and shearing, as needed so that the bone graft template and the secondanatomical object are in the same coordinate system. One example way inwhich preoperative surgical planning system 202 may perform therotation, translation, scaling, and shearing is using the OpenGLapplication programming interface (API); however, other ways in which toperform the rotation, translation, scaling, and shearing are possible.Also, once the bone graft template is in the coordinate system of thesecond anatomical object or before the bone graft template is in thecoordinate system of the second anatomical object, the surgeon mayrotate the bone graft template to view the bone graft template fromdifferent perspectives. Preoperative surgical planning system 202performing the above example operations of aligning coordinate system,rotating, and moving the bone graft template into the representation ofthe second anatomical object are non-limiting examples of preoperativesurgical planning system 202 determining information indicative of aplacement of the bone graft template within a representation of thesecond anatomical object based on the image data.

In the above example of preoperative surgical planning system 202determining information indicative of a placement of the bone grafttemplate within a representation of the second anatomical object basedon the image data, the surgeon performed “dragging and dropping”operations. In some examples, preoperative surgical planning system 202may be configured to determine information indicative of placement ofthe bone graft template within a representation of the second anatomicalobject based on the image data with little to no intervention from thesurgeon.

For example, preoperative surgical planning system 202 may align thebone graft template to the coordinate system of the second anatomicalobject. Preoperative surgical planning system 202 may then, based on thecoordinates of the bone graft template (e.g., coordinates along theboundary of the bone graft template) and coordinates of the secondanatomical object, move the bone graft template to be within therepresentation of the second anatomical object. For instance,preoperative surgical planning system 202 may rotate and shift the bonegraft template so that the bone graft template fits within therepresentation of the second anatomical object.

Accordingly, preoperative surgical planning system 202 may compare asize and shape of the bone graft template to the representation of thesecond anatomical object and determine information indicative of theplacement based on the comparison. In this manner, preoperative surgicalplanning system 202 may determine information indicative of placement ofthe bone graft template within a representation of the second anatomicalobject based on the image data.

In the above examples, the bone graft template is described as beingaligned with the coordinate system of the second anatomical object. Insome examples, the second anatomical object may be aligned with thecoordinate system of the bone graft template.

In some examples, preoperative surgical planning system 202 may beconfigured with criteria that preoperative surgical planning system 202uses when determining information indicative of placement of the bonegraft template within the representation of the second anatomicalobject. For example, preoperative surgical planning system 202 may beconfigured to determine bone density of the second anatomical objectsuch as based on U.S. Provisional Application No. 62/826,168 filed Mar.29, 2019. If the bone density is too low (e.g., below a threshold) incertain portions, preoperative surgical planning system 202 may notdetermine whether the bone graft template can be placed in the portionshaving low bone density.

As another example, preoperative surgical planning system 202 maydetermine whether a particular placement of the bone graft templatewould cause the bone graft template to cross a cortical wall within thesecond anatomical object. If the particular placement would cause thebone graft template to cross the cortical wall, then preoperativesurgical planning system 202 may determine that the particular placementis not a valid placement of the bone graft template.

As another example, preoperative surgical planning system 202 maydetermine whether a particular placement of the bone graft templatewould result in complicated surgery, preoperative surgical planningsystem 202 may determine that the particular placement is not a validplacement of the bone graft template. For example, if placement of thebone graft template in a particular location would result in the bonegraft not being accessible or require complicated surgery (e.g.,excessive shifting of bone, higher changes of complication, etc.) toaccess the bone graft, then preoperative surgical planning system 202may determine that the such placement of the bone graft template is notvalid.

There may be other criteria that preoperative surgical planning system202 may utilize when determining information indicative of placement ofthe bone graft template within the representation of the secondanatomical object. Preoperative surgical planning system 202 may beconfigured to use the above examples of the criteria and the additionalexamples of the criteria either alone or in any combination.

In some examples, preoperative surgical planning system 202 may beconfigured to output information indicative of whether the secondanatomical object is potentially suitable as a donor site for the bonegraft. For example, preoperative surgical planning system 202 mayutilize the various criteria to determine whether the bone grafttemplate can be placed in the second anatomical object. If there are novalid placements for the bone graft template, preoperative surgicalplanning system 202 may output information indicating that the secondanatomical object may not be suitable as a donor site. If there arevalid placements for the bone graft template, preoperative surgicalplanning system 202 may output information indicating that the secondanatomical object is suitable as a donor site.

In some examples, there may be multiple ways in which the bone grafttemplate can fit within the second anatomical object. Preoperativesurgical planning system 202 may output the various valid optionsindicating from where the bone graft can be extracted from the secondanatomical object. In some examples, preoperative surgical planningsystem 202 may rank the valid options. In some examples, preoperativesurgical planning system 202 may determine the best of the valid options(e.g., the location within the second anatomical object from where thebone graft can be removed with the greatest ease while minimizing impactto patient).

Preoperative surgical planning system 202 may be configured to outputinformation indicative of the placement of the bone graft templatewithin the representation of the second anatomical object. As oneexample, preoperative surgical planning system 202 may generateinformation used by visualization device 213 to render the bone grafttemplate within the representation of the second anatomical object atthe determined placement. As another example, preoperative surgicalplanning system 202 may generate coordinate values of the location ofthe bone graft template. There may be other examples of the informationthat preoperative surgical planning system 202 generates for outputtingthat is indicative of the placement of the bone graft template withinthe representation of the second anatomical object (e.g., donor site).

In some examples, preoperative surgical planning system 202 may beconfigured to generate pre-operative planning information based onplacement of the bone graft template within the representation of thesecond anatomical object. For example, the information indicative of theplacement of the bone graft template may include information indicativeof where the bone graft template is located within the representation ofthe second anatomical object. The bone graft template may thereforeprovide a visual indication of from where to extract the bone graft.

As one example, preoperative surgical planning system 202 may beconfigured to generate information indicative of a location within thesecond anatomical object from which the bone graft is to be extracted(e.g., that is to be cut for extracting the bone graft). Visualizationdevice 213 may display the location that is to be cut for extracting thebone graft. Visualization device 213 may display the locationpreoperatively and/or intraoperatively.

As one example, preoperative surgical planning system 202 may beconfigured to generate information indicative of an axis along which tothe second anatomical object for extracting the bone graft.Visualization device 213 may display the axis that is to be cut forextracting the bone graft. Visualization device 213 may display the axispreoperatively and/or intraoperatively.

As one example, preoperative surgical planning system 202 may beconfigured to generate information indicative of a maximum depth atwhich to cut the second anatomical object for extracting the bone graft.Visualization device 213 may display the maximum depth preoperativelyand/or intraoperatively.

As one example, preoperative surgical planning system 202 may beconfigured to generate information indicative of a types of tool toutilize to cut the second anatomical object for extracting the bonegraft. Visualization device 213 may display the types of toolspreoperatively and/or intraoperatively 1.

In the above examples, preoperative surgical planning system 202 isdescribed as performing various operations. In some examples, theoperations of preoperative surgical planning system 202 may be performedby processing device(s) 210. In some examples, some of the exampleoperations described above may be performed by preoperative surgicalplanning system 202 and some of the example operations described abovemay be performed by processing device(s) 210.

In this disclosure, processing circuitry may be considered as performingexample operations described in this disclosure. The processingcircuitry may be processing circuitry of preoperative surgical planningsystem 202 or may be processing device(s) 210. In some examples, theprocessing circuitry refers to the processing circuitry distributedbetween MR system 212 and preoperative surgical planning system 202, aswell as other processing circuitry in system 200.

FIG. 3 is a block diagram illustrating an example of computing systemconfigured to perform one or more examples described in this disclosure.FIG. 3 illustrates an example of computing system 300, and preoperativesurgical planning system 202 is an example of computing system 300.Examples of computing system 300 include various types of computingdevices, such as server computers, personal computers, smartphones,laptop computers, and other types of computing devices.

Computing system 300 includes processing circuitry 320, data storagesystem 304, and communication interface 306. Computing system 300 mayinclude additional components, such as a display, keyboard, etc., notshown in FIG. 3 for ease. Also, in some examples, computing system 300may include fewer components. For example, data storage system 304 maybe similar to storage system 206 of FIG. 2 and reside off of (e.g., beexternal to) computing system 300. However, data storage system 304 maybe part of computing system 300 as illustrated. Even in examples wheredata storage system 304 is external to computing system 300, computingsystem 300 may still include local memory for storing instructions forexecution by processing circuitry 302 and provide functionality forstoring data used by or generated by processing circuitry 302. When datastorage system 304 is the local memory, the amount of storage providedby data storage system 304 may less than storage system 206.

Examples of processing circuitry 302 include fixed-function processingcircuits, programmable circuits, or combinations thereof, such as one ormore digital signal processors (DSPs), general purpose microprocessors,application specific integrated circuits (ASICs), field programmablegate arrays (FPGAs), or other equivalent integrated or discrete logiccircuitry. Fixed-function circuits refer to circuits that provideparticular functionality and are preset on the operations that can beperformed. Programmable circuits refer to circuits that can programmedto perform various tasks and provide flexible functionality in theoperations that can be performed. For instance, programmable circuitsmay execute instructions specified by software or firmware that causethe programmable circuits to operate in the manner defined byinstructions of the software or firmware. Fixed-function circuits mayexecute software instructions (e.g., to receive parameters or outputparameters), but the types of operations that the fixed-functioncircuits perform are generally immutable.

Examples of data storage system 304 include RAM, ROM, EEPROM, CD-ROM orother optical disk storage, magnetic disk storage, or other magneticstorage devices, flash memory, or any other medium that can be used tostore data. In some examples, data storage system 304 may also storeprogram code in the form of instructions or data structures and that canbe accessed by processing circuitry 302 for execution.

Communication interface 306 refers to circuitry that allows computingsystem 300 to connect, wirelessly or with wired connection, with othercomponents. For instance, communication interface 306 provides thecircuitry that allows computing device 300 to transmit to and receivefrom network 208 of FIG. 2.

Processing circuitry 302 is an example of processing circuitryconfigured to perform one or more example techniques described in thisdisclosure. In some examples, such as where MR system 212 is configuredto perform various operations of preoperative surgical planning system202, processing device(s) 210 may include processing circuitry 302.Also, in some examples, the processing circuitry that is configured toperform the example operations described in this disclosure may includethe combination of processing circuitry 302, processing device(s) 210,and possibly one or more other processing circuitry. For example, FIG. 3is described with respect to processing circuitry 302.

For example, data storage system 304 may store image data for one ormore images of anatomical objects, and processing circuitry 302 mayaccess the image data from data storage system 304. Utilizing one ormore of the example techniques described above, processing circuitry 302may be configured to determine a bone graft template for a bone graft tobe connected to a first anatomical object, determine informationindicative of placement of the bone graft template within arepresentation of a second anatomical object based on the image data,and output the information indicative of the placement of the bone grafttemplate within the representation of the second anatomical object.

FIGS. 4 and 5 are conceptual diagrams illustrating examples of boneextraction for bone grafting. For example, FIGS. 4 and 5 illustrateexamples of bone extraction (e.g., harvesting) for bony increasedoffset-reverse shoulder arthroplasty (BIO-RSA). Other example proceduresthat may utilize the techniques described in this disclosure include theNorris technique. One example of the Norris technique is available fromNorris, Tom & Kelly, James & Humphrey, C. (2007). “Management of GlenoidBone Defects in Revision Shoulder Arthroplasty: A New Application of theReverse Total Shoulder Prosthesis.” Techniques in Shoulder & ElbowSurgery. 8. 37-46. 10.1097/BTE.0b013e318030d3b7.

As shown in FIG. 4, graft reaming tool 402 may be used to ream thesurface of humeral head 404 of humerus 400. The surgeon may connectgraft reaming tool 402 to a drill or other instrument and MR system 212may display virtual guidance to assist in reaming the surface of humeralhead 404. For instance, MR system 212 may display depth guidance toenable the surgeon to ream the surface of humeral head 404 to a targetdepth. As another example, MR system 212 may provide targeting guidance.For instance, MR system 212 may display one or both of a virtual markerthat identifies a center point or prescribed axis of the reaming and/oran indication of whether graft reaming tool 402 is aligned with theprescribed axis.

In this example, graft reaming tool 402 may be a cannulated reaming toolconfigured to be positioned and/or guided by a guide pin, such as guide406. In other examples, graft reaming tool 402 may not be cannulated andmay be guided without the assistance of a physical guide pin. Forinstance, MR system 212 may provide virtual guidance (e.g., depthguidance and/or targeting guidance such as a displayed virtual marker)to enable a surgeon to ream a graft from humeral head 404 without theuse of guide 406.

As shown in FIG. 5, a surgeon may attach mechanical guide 502 on humerus500 prior to performing a resection of humeral head 504. The surgeon mayadjust one or more components of mechanical guide 502 such that topsurface 506 of mechanical guide 502 is co-planar with anatomic neck 508of humerus 500 (for purposes of illustration, anatomic neck 508 isillustrated as a broken line). After attaching mechanical guide 502 tohumeral head 504 and adjusting the mechanical guide, the surgeon mayperform the resection of humeral head 504 by guiding a cutting tool(e.g., a blade of an oscillating saw) along top surface 506.

FIG. 6 is a conceptual diagram illustrating an example of the extractedbone grafted to glenoid bone. As shown in FIG. 6, bone graft 600,possibly taken from humeral head 404 (FIG. 4) or 504 (FIG. 5), isgrafted to glenoid 602. Stem 606 of prothesis 604 is inserted into bonegraft 600, as part of the BIO-RSA. Stem 606, with bone graft 600, isinserted into the humerus, as part of the BIO-RSA.

FIGS. 7A and 7B are conceptual diagrams illustrating examples of a bonegraft template within a representation of a donor site. FIGS. 7A and 7Billustrate bone graft template 702 and donor site (e.g., humeral head)704. For ease of illustration, bone graft template 702 and donor site704 is illustrated as two-dimensional images. Visualization device 213may be configured to display bone graft template 702 and donor site 704as three-dimensional images. However, visualization device 213 maydisplay bone graft template 702 and donor site 704 in two-dimensionalimages, such as images from different perspectives.

Processing circuitry (e.g., processing circuitry 302, processingdevice(s) 210, or a combination thereof) may be configured to determinebone graft template 702 for a bone graft to be connected to a firstanatomical object (e.g., glenoid bone 602 of FIG. 6). The processingcircuitry may utilize the various example techniques described above todetermine bone graft template 702. The processing circuitry may beconfigured to determine information indicative of placement of bonegraft template 702 within a representation of a second anatomical object(e.g., donor site 704) based on the image data utilizing the aboveexample techniques.

The processing circuitry may output information indicative of theplacement of bone graft template 702 within the representation of thesecond anatomical object (e.g., donor site 704). For example, FIGS. 7Aand 7B illustrate the rendering of bone graft template 702 within donorsite 704 from different perspectives. The graphical information used torender bone graft template 702 within donor site 704 is an example ofthe information that the processing circuitry outputs indicative of theplacement of bone graft template 702 within donor site 704.

FIG. 8 is a flowchart illustrating example methods of operations inaccordance with one or more example techniques described in thisdisclosure. The example techniques of FIG. 8 may be performed withprocessing circuitry (e.g., processing circuitry 302, processingdevice(s) 210, or a combination thereof).

The processing circuitry may determine a bone graft template for a bonegraft to be connected to a first anatomical object (800). As oneexample, the processing circuitry may be configured to receive input ofthe bone graft template (e.g., based on drawing from surgeon orselection of pre-generated bone graft template). As another example, theprocessing circuitry may be configured to determine a shape of the firstanatomical object, compare the shape of the first anatomical object to apre-morbid shape of the first anatomical object, and determine the bonegraft template based on the comparison.

The processing circuitry may determine information indicative ofplacement of the bone graft template within a representation of a secondanatomical object based on stored image data (802). As one example, theprocessing circuitry may be configured to receive information indicativeof placement of the bone graft template within the representation of thesecond anatomical object (e.g., based on “drag and drop” by surgeon). Asone example, the processing circuitry may compare a size and shape ofthe bone graft template to the representation of the second anatomicalobject and determine information indicative of the placement based onthe comparison.

The processing circuitry may output information indicative of theplacement of the bone graft template within the representation of thesecond anatomical object (804). As one example, processing circuitry isconfigured to output graphical information for rendering, for display,the bone graft template within the representation of the secondanatomical object. For instance, the processing circuitry may outputgraphical information for rendering, for display, the representation ofthe second anatomical object. In such examples, to receive informationindicative of the placement of the bone graft template, the processingcircuitry may be configured to receive information indicative of theplacement of the bone graft template within the displayed representationof the second anatomical object. To output the information indicative ofthe placement of the bone graft, the processing circuitry may beconfigured to output graphical information for rendering, for display,the bone graft template within the displayed represented of the secondanatomical object based on the received information indicative of theplacement of the bone graft template within the displayed representationof the second anatomical object.

In some examples, the processing circuitry may be configured to outputinformation indicative of whether the second anatomical object ispotentially suitable as a donor site for the bone graft for the firstanatomical object. For example, the processing circuitry may beconfigured with criteria to use to determine whether a placement of thebone graft template within the second anatomical object is valid. If theprocessing circuitry determines that there is no valid placement of thebone graft template in the second anatomical object, the processingcircuitry may output information indicating that the second anatomicalobject is not suitable for a donor site for the bone graft for the firstanatomical object.

The processing circuitry, may in some examples, recommend an alternativesite when no valid placement of the bone graft is found. For example,the processing circuitry may be preloaded with representations ofdifferent possible second anatomical objects (e.g., humeral head, iliaccrest, etc.). The processing circuitry may determine whether there arevalid placements of the bone graft in the different second anatomicalobjects, and identify potential locations that can be candidates for thesecond anatomical object. Rather than trying different possible secondanatomical objects, in some examples, the processing circuitry maydetermine valid placements in the different second anatomical objectsand identify which ones of the second anatomical objects can be donorsites. The processing circuitry may also provide a ranking of secondanatomical objects indicating which ones could be the best donor sites(e.g., in terms of ease of surgery and most available bone).

In some examples, the processing circuitry may generate pre-operativeplanning information based on placement of the bone graft templatewithin the representation of the second anatomical object (806). As oneexample, the processing circuitry is configured to generate informationindicative of a location within the second anatomical object from whichthe bone graft is to be extracted (e.g., that is to be cut forextracting the bone graft). As another example, the processing circuitryis configured to generate information indicative of an axis along whichto the second anatomical object for extracting the bone graft. Asanother example, the processing circuitry is configured to generateinformation indicative of a maximum depth at which to cut the secondanatomical object for extracting the bone graft. As another example, theprocessing circuitry is configured to generate information indicative ofa types of tool to utilize to cut the second anatomical object forextracting the bone graft.

While the techniques been disclosed with respect to a limited number ofexamples, those skilled in the art, having the benefit of thisdisclosure, will appreciate numerous modifications and variations therefrom. For instance, it is contemplated that any reasonable combinationof the described examples may be performed. It is intended that theappended claims cover such modifications and variations as fall withinthe true spirit and scope of the invention.

It is to be recognized that depending on the example, certain acts orevents of any of the techniques described herein can be performed in adifferent sequence, may be added, merged, or left out altogether (e.g.,not all described acts or events are necessary for the practice of thetechniques). Moreover, in certain examples, acts or events may beperformed concurrently, e.g., through multi-threaded processing,interrupt processing, or multiple processors, rather than sequentially.

In one or more examples, the functions described may be implemented inhardware, software, firmware, or any combination thereof. If implementedin software, the functions may be stored on or transmitted over as oneor more instructions or code on a computer-readable medium and executedby a hardware-based processing unit. Computer-readable media may includecomputer-readable storage media, which corresponds to a tangible mediumsuch as data storage media, or communication media including any mediumthat facilitates transfer of a computer program from one place toanother, e.g., according to a communication protocol. In this manner,computer-readable media generally may correspond to (1) tangiblecomputer-readable storage media which is non-transitory or (2) acommunication medium such as a signal or carrier wave. Data storagemedia may be any available media that can be accessed by one or morecomputers or one or more processors to retrieve instructions, codeand/or data structures for implementation of the techniques described inthis disclosure. A computer program product may include acomputer-readable medium.

By way of example, and not limitation, such computer-readable storagemedia can comprise RAM, ROM, EEPROM, CD-ROM or other optical diskstorage, magnetic disk storage, or other magnetic storage devices, flashmemory, or any other medium that can be used to store desired programcode in the form of instructions or data structures and that can beaccessed by a computer. Also, any connection is properly termed acomputer-readable medium. For example, if instructions are transmittedfrom a website, server, or other remote source using a coaxial cable,fiber optic cable, twisted pair, digital subscriber line (DSL), orwireless technologies such as infrared, radio, and microwave, then thecoaxial cable, fiber optic cable, twisted pair, DSL, or wirelesstechnologies such as infrared, radio, and microwave are included in thedefinition of medium. It should be understood, however, thatcomputer-readable storage media and data storage media do not includeconnections, carrier waves, signals, or other transitory media, but areinstead directed to non-transitory, tangible storage media. Disk anddisc, as used herein, includes compact disc (CD), laser disc, opticaldisc, digital versatile disc (DVD), floppy disk and Blu-ray disc, wheredisks usually reproduce data magnetically, while discs reproduce dataoptically with lasers. Combinations of the above should also be includedwithin the scope of computer-readable media.

Operations described in this disclosure may be performed by one or moreprocessors or processing circuitry, which may be implemented asfixed-function processing circuits, programmable circuits, orcombinations thereof, such as one or more digital signal processors(DSPs), general purpose microprocessors, application specific integratedcircuits (ASICs), field programmable gate arrays (FPGAs), or otherequivalent integrated or discrete logic circuitry. Fixed-functioncircuits refer to circuits that provide particular functionality and arepreset on the operations that can be performed. Programmable circuitsrefer to circuits that can programmed to perform various tasks andprovide flexible functionality in the operations that can be performed.For instance, programmable circuits may execute instructions specifiedby software or firmware that cause the programmable circuits to operatein the manner defined by instructions of the software or firmware.Fixed-function circuits may execute software instructions (e.g., toreceive parameters or output parameters), but the types of operationsthat the fixed-function circuits perform are generally immutable.Accordingly, the terms “processor” and “processing circuitry,” as usedherein may refer to any of the foregoing structures or any otherstructure suitable for implementation of the techniques describedherein.

Various examples have been described. These and other examples arewithin the scope of the following claims.

1: A system for bone graft selection in orthopedic surgery, the systemcomprising: a memory configured to store image data for one or moreimages of anatomical objects; and processing circuitry configured to:determine a bone graft template for a bone graft to be connected to afirst anatomical object; determine information indicative of placementof the bone graft template within a representation of a secondanatomical object based on the image data; and output the informationindicative of the placement of the bone graft template within therepresentation of the second anatomical object. 2: The system of claim1, wherein to determine the bone graft template, the processingcircuitry is configured to receive input of the bone graft template. 3:The system of claim 1, wherein to determine the bone graft template, theprocessing circuitry is configured to: determine a shape of the firstanatomical object; compare the shape of first anatomical object to apre-morbid shape of the first anatomical object; and determine the bonegraft template based on the comparison. 4: The system of claim 1,wherein to determine information indicative of placement, the processingcircuitry is configured to receive information indicative of placementof the bone graft template within the representation of the secondanatomical object, and wherein to output information indicative of theplacement, the processing circuitry is configured to output graphicalinformation for rendering, for display, the bone graft template withinthe representation of the second anatomical object. 5: The system ofclaim 4, wherein the processing circuitry is configured to: outputgraphical information for rendering, for display, the representation ofthe second anatomical object, wherein to receive information indicativeof the placement of the bone graft template, the processing circuitry isconfigured to receive information indicative of the placement of thebone graft template within the displayed representation of the secondanatomical object, and wherein to output the information indicative ofthe placement of the bone graft, the processing circuitry is configuredto output graphical information for rendering, for display, the bonegraft template within the displayed represented of the second anatomicalobject based on the received information indicative of the placement ofthe bone graft template within the displayed representation of thesecond anatomical object. 6: The system of claim 1, wherein to determineinformation indicative of placement, the processing circuitry isconfigured to: compare a size and shape of the bone graft template tothe representation of the second anatomical object; and determineinformation indicative of the placement based on the comparison, andwherein to output information indicative of the placement, theprocessing circuitry is configured to output graphical information forrendering, for display, the bone graft template within therepresentation of the second anatomical object. 7: The system of claim1, wherein the processing circuitry is configured to output informationindicative of whether the second anatomical object is potentiallysuitable as a donor site for the bone graft for the first anatomicalobject. 8: The system of claim 1, wherein the processing circuitry isconfigured to generate pre-operative planning information based onplacement of the bone graft template within the representation of thesecond anatomical object. 9: The system of claim 8, wherein to generatepre-operative planning information, the processing circuitry isconfigured to generate information indicative of a location within thesecond anatomical object from which the bone graft is to be extracted.10: The system of claim 8, wherein to generate pre-operative planninginformation, the processing circuitry is configured to generateinformation indicative of an axis along which to the second anatomicalobject for extracting the bone graft. 11: The system of claim 8, whereinto generate pre-operative planning information, the processing circuitryis configured to generate information indicative of a maximum depth atwhich to cut the second anatomical object for extracting the bone graft.12: The system of claim 8, wherein to generate pre-operative planninginformation, the processing circuitry is configured to generateinformation indicative of a types of tool to utilize to cut the secondanatomical object for extracting the bone graft. 13: A method for bonegraft selection in orthopedic surgery, the method comprising:determining a bone graft template for a bone graft to be connected to afirst anatomical object; determining information indicative of placementof the bone graft template within a representation of a secondanatomical object based on image data for one or more images ofanatomical objects; and outputting the information indicative of theplacement of the bone graft template within the representation of thesecond anatomical object. 14: The method of claim 13, whereindetermining the bone graft template comprises receiving input of thebone graft template. 15: The method of claim 13, wherein determining thebone graft template comprises: determining a shape of the firstanatomical object; comparing the shape of first anatomical object to apre-morbid shape of the first anatomical object; and determining thebone graft template based on the comparison. 16: The method of claim 13,determining information indicative of placement comprises receivinginformation indicative of placement of the bone graft template withinthe representation of the second anatomical object, and outputtinginformation indicative of the placement comprises outputting graphicalinformation for rendering, for display, the bone graft template withinthe representation of the second anatomical object. 17: The method ofclaim 16, further comprising: outputting graphical information forrendering, for display, the representation of the second anatomicalobject, wherein receiving information indicative of the placement of thebone graft template comprises receiving information indicative of theplacement of the bone graft template within the displayed representationof the second anatomical object, and outputting the informationindicative of the placement of the bone graft comprises outputtinggraphical information for rendering, for display, the bone grafttemplate within the displayed represented of the second anatomicalobject based on the received information indicative of the placement ofthe bone graft template within the displayed representation of thesecond anatomical object. 18: The method of claim 13, whereindetermining information indicative of placement comprises: comparing asize and shape of the bone graft template to the representation of thesecond anatomical object; and determining information indicative of theplacement based on the comparison, and wherein outputting informationindicative of the placement comprises outputting graphical informationfor rendering, for display, the bone graft template within therepresentation of the second anatomical object. 19: The method of claim13, further comprising outputting information indicative of whether thesecond anatomical object is potentially suitable as a donor site for thebone graft for the first anatomical object. 20: The method of claim 13,further comprising generating pre-operative planning information basedon placement of the bone graft template within the representation of thesecond anatomical object. 21: The method of claim 20, wherein generatingpre-operative planning information comprises generating informationindicative of a location within the second anatomical object from whichthe bone graft is to be extracted. 22: The method of claim 20, whereingenerating pre-operative planning information comprises generatinginformation indicative of an axis along which to the second anatomicalobject for extracting the bone graft. 23: The method of claim 20,wherein generating pre-operative planning information comprisesgenerating information indicative of a maximum depth at which to cut thesecond anatomical object for extracting the bone graft. 24: The methodof claim 20, wherein generating pre-operative planning informationcomprises generating information indicative of a types of tool toutilize to cut the second anatomical object for extracting the bonegraft. 25: A system for bone graft selection in orthopedic surgery, thesystem comprising: means for determining a bone graft template for abone graft to be connected to a first anatomical object; means fordetermining information indicative of placement of the bone grafttemplate within a representation of a second anatomical object based onimage data for one or more images of anatomical objects; and means foroutputting the information indicative of the placement of the bone grafttemplate within the representation of the second anatomical object. 26:The system of claim 25, wherein the means for determining informationindicative of placement comprises: means for comparing a size and shapeof the bone graft template to the representation of the secondanatomical object; and means for determining information indicative ofthe placement based on the comparison, and wherein the means foroutputting information indicative of the placement comprises means foroutputting graphical information for rendering, for display, the bonegraft template within the representation of the second anatomicalobject. 27: A computer-readable storage medium storing instructions thatwhen executed cause one or more processors to: determine a bone grafttemplate for a bone graft to be connected to a first anatomical object;determine information indicative of placement of the bone graft templatewithin a representation of a second anatomical object based on imagedata for one or more images of anatomical objects; and output theinformation indicative of the placement of the bone graft templatewithin the representation of the second anatomical object. 28: Thecomputer-readable storage medium of claim 27, further comprising whereinthe instructions that cause the one or more processors to determineinformation indicative of placement comprise instructions that cause theone or more processors to: compare a size and shape of the bone grafttemplate to the representation of the second anatomical object; anddetermine information indicative of the placement based on thecomparison, and wherein the instructions that cause the one or moreprocessors to output information indicative of the placement compriseinstructions that cause the one or more processors to output graphicalinformation for rendering, for display, the bone graft template withinthe representation of the second anatomical object.